Proofs of results stated in the main text are provided in Appendix A. Additional experimental results, including coverage plots, are provided in Appendix B. Therefore, the proof is completed by continuous mapping. Applying Taylor's theorem using the Lagrange form of the remainder, we have that, for some random From the proof of Eq. (7), we know that, with probability tending to 1, Figure 1 shows the fitted mean and covariance on a single draw of the quadratic dataset. In this section we provide details for the experimental setup used in the paper. The visualized covariance matrices were projecting to ensure positive semi-definiteness. In this section we present the multivariate algorithm for finite-difference IDM (FDIDM).

The supplementary material is outlined as follows. Section D details how we apply POLCO in the robotics tasks. Concretely, this means that safety constraints in our environment are specified via language. The environment is procedurally generated. We collected natural language constraints in a two-step process.